Autonomous Apple Fruitlet Sizing with Next Best View Planning

In this paper, we present a next-best-view planning approach to autonomously size apple fruitlets. State-of-the-art viewpoint planners in agriculture are designed to size large and more sparsely populated fruit. They rely on lower resolution maps and sizing methods that do not generalize to smaller fruit sizes. To overcome these limitations, our method combines viewpoint sampling around semantically labeled regions of interest, along with an attention-guided information gain mechanism to more strategically select viewpoints that target the small fruits' volume. Additionally, we integrate a dual-map representation of the environment that is able to both speed up expensive ray casting operations and maintain the high occupancy resolution required to informatively plan around the fruit. When sizing, a robust estimation and graph clustering approach is introduced to associate fruit detections across images. Through simulated experiments, we demonstrate that our viewpoint planner improves sizing accuracy compared to state of the art and ablations. We also provide quantitative results on data collected by a real robotic system in the field

Occlusion Reasoning for Skeleton Extraction of Self-Occluded Tree Canopies

In this work, we present a method to extract the skeleton of a self-occluded tree canopy by estimating the unobserved structures of the tree. A tree skeleton compactly describes the topological structure and contains useful information such as branch geometry, positions and hierarchy. This can be critical to planning contact interactions for agricultural manipulation, yet is difficult to gain due to occlusion by leaves, fruits and other branches. Our method uses an instance segmentation network to detect visible trunk, branches, and twigs. Then, based on the observed tree structures, we build a custom 3D likelihood map in the form of an occupancy grid to hypothesize on the presence of occluded skeletons through a series of minimum cost path searches. We show that our method outperforms baseline methods in highly occluded scenes, demonstrated through a set of experiments on a synthetic tree dataset. Qualitative results are also presented on a real tree dataset collected from the field.Comment: 7 pages, 10 figures, submitted to ICRA 202

Relational database of treatment planning system information

The purpose of the present work was to develop a relational database and associated applications to facilitate retrospective review of data present in radiation treatment plans. The data source was a commercial radiation treatment planning system (Pinnacle3, Philips Medical Systems, Milpitas CA), which is specifically characterized by an open data storage format and internal scripting capability. The database is an open-source, relational database (PostgreSQL, PostgreSQL Global Development Group, http://www.postgresql.org). The data is presented through a web interface in addition to being fully query-accessible using standard tools. A database schema was created to organize the large collection of parameters used to generate treatment plans as well as the parameters that characterized these plans. The system was implemented through a combination of the treatment planning systems internal scripting language and externally executed code. Data is exported in a way that is transparent to the user, through integration into an existing and routinely-used process. The system has been transparently incorporated into our radiation treatment planning workflow. The website-based database interface has allowed users with minimal training to extract information from the database

3D Skeletonization of Complex Grapevines for Robotic Pruning

Robotic pruning of dormant grapevines is an area of active research in order to promote vine balance and grape quality, but so far robotic efforts have largely focused on planar, simplified vines not representative of commercial vineyards. This paper aims to advance the robotic perception capabilities necessary for pruning in denser and more complex vine structures by extending plant skeletonization techniques. The proposed pipeline generates skeletal grapevine models that have lower reprojection error and higher connectivity than baseline algorithms. We also show how 3D and skeletal information enables prediction accuracy of pruning weight for dense vines surpassing prior work, where pruning weight is an important vine metric influencing pruning site selection.Comment: 6 pages, IROS 2023 Computer Vision for Automatio

Approximate Matrix Diagonalization for Use in Distributed Control Networks

Distributed control networks are rapidly emerging as aviable and important alternative to centralized control. In a typical distributed control network, a number of spatially distributed nodescomposed of "smart" sensors and actuators are used to take measurements and apply control inputs to some physical plant. The nodes have local processing power and the ability to communicate with the other nodes via a network. The challenge is to compute and implement a feedback law for the resulting MIMO system in a distributed manner on the network.Our approach to this problem is based on plant diagonalization.To do this, we search for basis transformations for the vector of outputs coming from the sensors and the vector of inputs applied to the actuators so that, in the new bases, the MIMOsystem becomes a collection of decoupled SISO systems.This formulation provides a number of advantages for the synthesis and implementation of a feedback control law,particularly for systems where the number of inputs and outputs is large. Of course, in order for this idea to be feasible,the required basis transformations must have properties which allow them to be implemented on a distributed control network. Namely, they must be computed in a distributed manner which respects the spatial distribution of the data(to reduce communication overhead) and takes advantage of the massive parallel processing capability of the network (to reduce computation time). In this thesis, we present some tools which can be used to find suitable transforms which achieve "approximate"plant diagonalization. We begin by showing how to search the large collection of orthogonal transforms which are contained in the wavelet packet to find the one which most nearly, or approximately, diagonalizes a given real valued matrix.Wavelet packet transforms admit a natural distributed implementation,making them suitable for use on a control network.We then introduce a class of linear operators called recursive orthogonal transforms (ROTs) which we have developed specifically for the purpose of signal processing on distributed control networks. We show how to use ROTs to approximately diagonalize fixed real and complex matricesas well as transfer function matrices which exhibit a spatial invariance property. Numerical examples of allproposed diagonalization methods are presented and discussed

Towards Robotic Tree Manipulation: Leveraging Graph Representations

There is growing interest in automating agricultural tasks that require intricate and precise interaction with specialty crops, such as trees and vines. However, developing robotic solutions for crop manipulation remains a difficult challenge due to complexities involved in modeling their deformable behavior. In this study, we present a framework for learning the deformation behavior of tree-like crops under contact interaction. Our proposed method involves encoding the state of a spring-damper modeled tree crop as a graph. This representation allows us to employ graph networks to learn both a forward model for predicting resulting deformations, and a contact policy for inferring actions to manipulate tree crops. We conduct a comprehensive set of experiments in a simulated environment and demonstrate generalizability of our method on previously unseen trees. Videos can be found on the project website: https://kantor-lab.github.io/tree_gnnComment: 7 pages, 10 figure